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ABSTRACT 

Shannon [40] sought security against the attacker with un- 
hmited computational powers: if an information source con- 
veys some information, then Shannon's attacker will surely 
extract that information. DiiBe and Hellman TSl refined 
Shannon's attacker model by taking into account the fact 
that the real attackers are computationally limited. This 
idea became one of the greatest new paradigms in computer 
science, and led to modern cryptography. 

Shannon also sought security against the attacker with un- 
limited logical and observational powers, expressed through 
the maxim that "the enemy knows the system". This view 
is still endorsed in cryptography. The popular formulation, 
going back to Kerckhoffs [24l , is that "there is no security by 
obscurity", meaning that the algorithms cannot be kept ob- 
scured from the attacker, and that security should only rely 
upon the secret keys. In fact, modern cryptography goes 
even further than Shannon or Kerckhoffs in tacitly assum- 
ing that if there is an algorithm that can break the system, 
then the attacker will surely find that algorithm. The at- 
tacker is not viewed as an omnipotent computer any more, 
but he is still construed as an omnipotent programmer. The 
ongoing hackers' successes seem to justify this view. 

So the Diffie-Hellman step from unlimited to limited com- 
putational powers has not been extended into a step from 
unlimited to limited logical or programming powers. Is the 
assumption that all feasible algorithms will eventually be 
discovered and implemented really different from the as- 
sumption that everything that is computable will eventually 
be computed? The present paper explores some ways to re- 
fine the current models of the attacker, and of the defender, 
by taking into account their limited logical and program- 
ming powers. If the adaptive attacker actively queries the 
system to seek out its vulnerabilities, can the system gain 
some security by actively learning attacker's methods, and 
adapting to them? 



1. INTRODUCTION 

New paradigms change the world. In computer science, they 
often sneak behind researchers' backs: the grand visions of- 
ten frazzle into minor ripples (like the fifth generation of 
programming languages), whereas some modest goals en- 
gender tidal waves with global repercussions (like moving 
the cursor by a device with wheels, or connecting remote 
computers). So it is not easy to conjure a new paradigm 
when you need it. 

Perhaps the only readily available method to generate new 
paradigms at leisure is by disputing the obvious. Just in 
case, I question on this occasion not one, but two generally 
endorsed views: 



• Kerckhoffs Principle that there is no security by ob- 
scurity, and 

• Fortification Principle that the defender has to defend 
all attack vectors, whereas the attacker only needs to 
attack one. 



To simplify things a little, I argue that these two princi- 
ples as related. The Kerckhoffs Principle demands that a 
system should withstand attackers unhindered probing. In 
the modern security definitions, this is amplified to the re- 
quirement that the system should resist a family of attacks, 
irrespective of the details of their algorithms. The adaptive 
attackers are thus allowed to query the system, whereas the 
system is not allowed to query the attackers. The resulting 
information asymmetry makes security look like a game bi- 
ased in favor of the attackers. The Fortification Principle is 
an expression of that asymmetry. In economics, information 
asymmetry has been recognized as a fundamental problem, 
worth the Nobel Prize in Economics for 2001 [MEIHl]. In 
security research, the problem does not seem to have been 
explicitly addressed, but there is, of course, no shortage of 
efforts to realize security by obscurity in practice — albeit 
without any discernible method. Although the practices of 
analyzing the attackers and hiding the systems are hardly 
waiting for anyone to invent a new paradigm, I will pur- 
sue the possibility that a new paradigm might be sneaking 
behind our backs again, like so many old paradigms did. 



Outline of the paper 

While I am on the subject of security paradigms, I decided 
to first spell out a general overview of the old ones. An 



attempt at this is in Sec. [2] It is surely incomplete, and 
perhaps wrongheaded, but it may help a little. It is difficult 
to communicate about the new without an agreement about 
the old. Moreover, it will be interesting to hear not only 
whether my new paradigms are new, but also whether my 
old paradigms are old. 

The new security paradigm arising from the slogan "Know 
your enemy" is discussed in Sec.[3l Of course, security engi- 
neers often know their enemies, so this is not much of a new 
paradigm in practice. But security researchers often require 
that systems should be secure against universal families of 
attackers, without knowing anything about who the enemy 
is at any particular moment. With respect to such static 
requirements, a game theoretic analysis of dynamics of se- 
curity can be viewed as an almost-new paradigm (with few 
previous owners). In Sec l3.1l I point to the practical devel- 
opments that lead up to this paradigm, and then in Sec. 13.21 
I describe the game of attack vectors, which illustrates it. 
This is a very crude view of security process as a game of 
incomplete information. I provide a simple pictorial analy- 
sis of the strategic interactions in this game, which turn out 
to be based on acquiring information about the opponent's 
type and behavior. A sketch of a formal model of security 
games of incomplete information, and of the game of attack 
vectors, is given in Appendix [XI 

A brand new security paradigm of "Applied security by ob- 
scurity" is described in Sec. It is based on the idea of 
logical complexity of programs, which leads to one way pro- 
gramming similarly like computational complexity led to one 
way computations. If achieved, one way programming will 
be a powerful tool in security games. 

A final attempt at a summary, and some comments about 
the future research, and the pitfalls, are given in Sec. [S] 



Related work 

The two new paradigms offer two new tools for the security 
toolkit: games of incomplete information, and algorithmic 
information theory. 

Game theoretic techniques have been used in applied secu- 
rity for a long time, since there a need for strategic reasoning 
often arises in practice. A typical example from the early 
days is [12] , where games of imperfect information were used. 
Perhaps the simplest more recent game based model are the 
attack-defense trees, which boil down to zero-sum extensive 
games [271 . Another application of games of imperfect infor- 
mation appeared, e.g., in a previous edition of this confer- 
ence [31]. Conspicuously, games of incomplete information 
do not seem to have been used, which seems appropriate 
since they analyze how players keep each other in obscurity. 
The coalgebraic presentation of games and response rela- 
tions, presented in the Appendix, is closely related with the 
formalism used in |36) . 

The concept of logical complexity, proposed in Sec. [4] is 
based on the ideas of algorithmic information theory [261 
I41j in general, and in particular on the idea of logical depth 
[111 1281 r?]. I propose to formalize logical complexity by lift- 
ing logical depth from the Godel-Kleene indices to program 



specifications [21] H [TO] [15] [H [3?]. The underlying idea 
that a Godel-Kleene index of a program can be viewed as 
its "explanation" goes back to Kleene's idea of realizability 
[25| and to Solomonoff 's formalization of inductive inference 

2. OLD SECURITY PARADIGMS 

Security means many things to many people. For a software 
engineer, it often means that there are no buffer overflows or 
dangling pointers in the code. For a cryptographer, it means 
that any successful attack on the cypher can be reduced 
to an algorithm for computing discrete logarithms, or to 
integer factorization. For a diplomat, security means that 
the enemy cannot read the confidential messages. For a 
credit card operator, it means that the total costs of the 
fraudulent transactions and of the measures to prevent them 
are low, relative to the revenue. For a bee, security means 
that no intruder into the beehive will escape her sting. . . 

Is it an accident that all these different ideas go under the 
same name? What do they really have in common? They are 
studied in different sciences, ranging from computer science 
to biology, by a wide variety of different methods. Would it 
be useful to study them together? 



2.1 What is security? 

If all avatars of security have one thing in common, it is 
surely the idea that there are enemies and potential attack- 
ers out there. All security concerns, from computation to 
politics and biology, come down to averting the adversarial 
processes in the environment, that are poised to subvert the 
goals of the system. There are, for instance, many kinds of 
bugs in software, but only those that the hackers use are a 
security concern. 

In all engineering disciplines, the system guarantees a func- 
tionality, provided that the environment satisfies some as- 
sumptions. This is the standard assume- guarantee format 
of the engineering correctness statements. Such statements 
are useful when the environment is passive, so that the as- 
sumptions about it remain valid for a while. The essence of 
security engineering is that the environment actively seeks 
to invalidate system's assumptions. 

Security is thus an adversarial process. In all engineering 
disciplines, failures usually arise from engineering errors and 
noncompliance. In security, failures arise in spite of the com- 
pliance with the best engineering practices of the moment. 
Failures are the first class citizens of security: every key has 
a lifetime, and in a sense, every system too. For all ma- 
jor software systems, we normally expect security updates, 
which usually arise from attacks, and often inspire them. 

2.2 Where did security come from? 

The earliest examples of security technologies are found among 
the earliest documents of civilization. Fig. [T] shows secu- 
rity tokens with a tamper protection technology from almost 
6000 years ago. Fig[2] depicts the situation where this tech- 
nology was probably used. Alice has a lamb and Bob has 
built a secure vault, perhaps with multiple security levels. 
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Figure 1: Tamper protection from 3700 BC 
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Figure 2: To withdraw her sheep from Bob's 
secure vault, Alice submits a tamper- proof 
token from Fig. [ij 



spacious enough to store both Bob's and Ahce's assets. For 
each of Ahce's assets deposited in the vault, Bob issues a 
clay token, with an inscription identifying the asset. Alice's 
tokens are then encased into a bulla, a round, hollow "en- 
velope" of clay, which is then baked to prevent tampering. 
When she wants to withdraw her deposits, Alice submits her 
bulla to Bob, he breaks it, extracts the tokens, and returns 
the goods. Alice can also give her bulla to Carol, who can 
also submit it to Bob, to withdraw the goods, or pass on 
to Dave. Bullae can thus be traded, and they facilitate ex- 
change economy. The tokens used in the bulla evolved into 
the earliest forms of money, and the inscriptions on them 
led to the earliest numeral systems, as well as to Sumerian 
cuneiform script, which was one of the earliest alphabets. 
Security thus predates literature, science, mathematics, and 
even money. 

2.3 Where is security going? 

Through history, security technologies evolved gradually, serv- 
ing the purposes of war and peace, protecting public re- 
sources and private property. As computers pervaded all 
aspects of social life, security became interlaced with com- 
putation, and security engineering came to be closely related 
with computer science. The developments in the realm of 
security are nowadays inseparable from the developments in 
the realm of computation. The most notable such develop- 
ment is, of course, cyber space. 

Paradigms of computation 

In the beginning, engineers built computers, and wrote pro- 
grams to control computations. The platform of computa- 
tion was the computer, and it was used to execute algorithms 
and calculations, allowing people to discover, e.g., fractals, 
and to invent compilers, that allowed them to write and exe- 
cute more algorithms and more calculations more efficiently. 
Then the operating system became the platform of compu- 
tation, and software was developed on top of it. The era of 
personal computing and enterprise software broke out. And 
then the Internet happened, followed by cellular networks, 
and wireless networks, and ad hoc networks, and mixed net- 
works. Cyber space emerged as the distance-free space of 
instant, costless communication. Nowadays software is de- 
veloped to run in cyberspace. The Web is, strictly speaking, 
just a software system, albeit a formidable one. A botnet is 



also a software system. As social space blends with cyber 
space, many social (business, collaborative) processes can be 
usefully construed as software systems, that ran on social 
networks as hardware. Many social and computational pro- 
cesses become inextricable. Table [1] summarizes the crude 
picture of the paradigm shifts which led to this remarkable 
situation. 

But as every person got connected to a computer, and every 
computer to a network, and every network to a network of 
networks, computation became interlaced with communica- 
tion, and ceased to be programmable. The functioning of 
the Web and of web applications is not determined by the 
code in the same sense as in a traditional software system: 
after all, web applications do include the human users as a 
part of their runtime. The fusion of social and computa- 
tional processes in cyber-social space leads to a new type of 
information processing, where the purposeful program ex- 
ecutions at the network nodes are supplemented by spon- 
taneous data-driven evolution of network links. While the 
network emerges as the new computer, data and metadata 
become inseparable, and a new type of security problems 
arises. 



Paradigms of security 

In early computer systems, security tasks mainly concerned 
sharing of the computing resources. In computer networks, 
security goals expanded to include information protection. 
Both computer security and information security essentially 
depend on a clear distinction between the secure areas, and 
the insecure areas, separated by a security perimeter. Secu- 
rity engineering caters for computer security and for infor- 
mation security by providing the tools to build the security 
perimeter. In cyber space, the secure areas are separated 
from the insecure areas by the "walls" of cryptography; and 
they are connected by the "gates" of cryptographic proto- 
colsQ But as networks of computers and devices spread 
through physical and social spaces, the distinctions between 

^This is, of course, a blatant oversimplification, as are many 
other statements I make. In a sense, every statement is an 
oversimplification of reality, abstracting away the matters 
deemed irrelevant. The gentle reader is invited to squint 
whenever any of the details that I omit do seem relevant, 
and add them to the picture. The shape of a forest should 
not change when some trees are enhanced. 
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TJllddlG CLQCS 


modem times 


platform 


computer 


operating system 


network 


applications 


Quicksort, compilers 


MS Word, Oracle 


WWW, botnets 


requirements 


correctness, termination 


liveness, safety 


trust, privacy 


tools 


programming languages 


specification languages 


scripting languages 



Table 1: Paradigm shifts in computation 



age 


middle ages 


modern times 


postmodern times 


space 


computer center 


cyber space 


cyber-social space 


assets 


computing resources 


information 


public and private resources 


requirements 


availability, authorization 


integrity, confidentiality 


trust, privacy 


tools 


locks, tokens, passwords 


cryptography, protocols 


mining and classification 



Table 2: Paradigm shifts in security 



the secure and the insecure areas become blurred. And in 
such areas of cyber-social space, information processing does 
not yield to programming, and cannot be secured just by 
cryptography and protocols. What else is there? 



3. A SECOND-HAND BUT ALMOST-NEW 
SECURITY PARADIGM: KNOW YOUR 
ENEMY 

3.1 Security beyond architecture 

Let us take a closer look at the paradigm shift to postmodern 
cyber security in Tabled It can be illustrated as the shift 
from Fig. |3]to Fig. |4] The fortification in Fig. [3] represents 
the view that security is in essence an architectural task. 
A fortress consists of walls and gates, separating the secure 
area within from the insecure area outside. The boundary 
between these two areas is the security perimeter. The se- 
cure area may be further subdivided into the areas of higher 
security and the areas of lower security. In cyber space, as 
we mentioned, the walls are realized using crypto systems, 
whereas the gates are authentication protocols. But as ev- 
ery fortress owner knows , the walls and the gates are not 
enough for security: you also need some soldiers to defend it, 
and some weapons to arm the soldiers, and some craftsmen 
to build the weapons, and so on. Moreover, you also need 
police and judges to maintain security within the fortress. 
They take care for the dynamic aspects of security. These 
dynamic aspects arise from the fact that sooner or later, 
the enemies will emerge inside the fortress: they will scale 
the walls at night (i.e. break the crypto), or sneak past the 
gatekeepers (break the protocols), or build up trust and en- 
ter with honest intentions, and later defect to the enemy; 
or enter as moles, with the intention to strike later. In any 
case, security is not localized at the security perimeters of 
Fig. |3] but evolves in-depth, like on Fig. |4l through social 
processes, like trust, privacy, reputation, influence. 

In summary, besides the methods to keep the attackers out, 
security is also concerned with the methods to deal with 




Figure 3: Static security 




Figure 4: Dynamic security 



the attackers once they get in. Security researchers have 
traditionally devoted more attention to the former family of 
methods. Insider threats have attracted a lot of attention 
recently, but a coherent set of research methods is yet to 
emerge. 

Interestingly, though, there is a sense in which security be- 
comes an easier task when the attacker is in. Although un- 
intuitive at the first sight, this idea becomes natural when 
security processes are viewed in a broad context of the infor- 
mation flows surrounding them (and not only with respect 
to the data designated to be secret or private). To view 
security processes in this broad context, it is convenient to 
model them as games of incomplete information |4], where 
the players do not have enough information to predict the 
opponent's behavior. For the moment, let me just say that 
the two families of security methods (those to keep the at- 
tackers out, and those to catch them when they are in) cor- 
respond to two families of strategies in certain games of in- 
complete information, and turn out to have quite different 
winning odds for the attacker, and for defender. In fact, 
they have the opposite winning odds. 

In the fortress mode, when the defenders' goal is to keep the 
attackers out, it is often observed that the attackers only 
need to find one attack vector to enter the fortress, whereas 
the defenders must defend all attack vectors to prevent them. 
When the battle switches to the dynamic mode, and the de- 
fense moves inside, then the defenders only need to find one 
marker to recognize and catch the attackers, whereas the 
attackers must cover all their markers. This strategic ad- 
vantage is also the critical aspect of the immune response, 
where the invading organisms are purposely sampled and 
analyzed for chemical markers. Some aspects of this obser- 
vation have, of course, been discussed within the framework 
of biologically inspired security. Game theoretic modeling 
seems to be opening up a new dimension in this problem 
space. We present a sketch to illustrate this new technical 
and conceptual direction. 

3.2 The game of attack vectors 

Arena. Two players, the attacker A and the defender D, 
battle for some assets of value to both of them. They are 
given equal, disjoint territories, with the borders of equal 
length, and equal amounts of force, expressed as two vector 
fields distributed along their respective borders. The players 
can redistribute the forces and move the borders of their 
territories. The territories can thus take any shapes and 
occupy any areas where the players may move them, obeying 
the constraints that 

(i) the length of the borders of both territories must be 
preserved, and 

(ii) the two territories must remain disjoint, except that 
they may touch at the borders. 

It is assumed that the desired asset O is initially held by the 
defender D. Suppose that storing this asset takes an area 
of size 6. Defender's goal is thus to maintain a territory po 
with an area / po > d. Attacker's goal is to decrease the 



size of pd below 9, so that the defender must release some of 
the asset Q. To achieve this, the attacker A must bring hifl 
forces to defender D's borders, and push into his territory. 
A position in the game can thus be something like Fig. (5] 

Game. At each step in the game, each player makes a move 
by specifying a distribution of his forces along his borders. 
Both players are assumed to be able to redistribute their 
forces with equal agility. The new force vectors meet at 
the border, they add up, and the border moves along the 
resulting vector. So if the vectors are, say, in the opposite 
directions, the forces subtract and the border is pushed by 
the greater vector. 

The players observe each other's positions and moves in two 
ways: 

(a) Each player knows his own moves, i.e. distributions, 
and sees how his borders change. From the change in 
the previous move, he can thus derive the opponent's 
current distribution of the forces along the common 
part of the border. 

(b) Each player sees all movement in the areas enclosed 
enclosed within his territory, i.e. observes any point on 
a straight line between any two points that he controls. 
That means that each player sees the opponent's next 
move at all points that lie within the convex hull of his 
territory, which we call range. 

According to (b), the position in Fig. fallows A to see D's 
next move. D, on the other hand only gets to know A's move 
according to (a), when his own border changes. This depicts, 
albeit very crudely, the information asymmetry between the 
attacker and the defender. 

Question. How should rational players play this game? 
3.2.1 Fortification strategy 

Goals. Since each player's total force is divided by the 
length of his borders, the maximal area defensible by a given 
force has the shape of a disk. All other shapes with the 
boundaries of the same length enclose smaller areas. So D's 
simplest strategy is to acquire and maintain the smallest disk 
shaped territory of size This is the fortification strategy: 
D only responds to A's moves. 

A's goal is, on the other hand, to create "dents" in D's ter- 
ritory pd, since the area of po decreases most when its con- 
vexity is disturbed. If a dent grows deep enough to reach 
across po, or if two dents in it meet, then po disconnects 

^I hope no one minds that I will be using "he" for both 
ylttacker and Defender, in an attempt to avoid distracting 
connotations. 

■^This is why the core of a medieval fortification was a round 
tower with a thick wall and a small space inside. The fortress 
itself often is not round, because the environment is not flat, 
or because the straight walls were easier to build; but it 
is usually at least convex. Later fortresses, however, had 
protruding towers — to attack the attacker. Which leads us 
beyond the fortification strategy. . . 



Figure 5: Fortification Figure 6: Honeypot 



Figure 7: Sampling 



Figure 8: Adaptation 



in two components. Given a constant length of the border, 
it is easy to see that the size of the enclosed area decreases 
exponentially as it gets broken up. In this way, the area 
enclosed by a border of given length can be made arbitrarily 
small. 

But how can A create dents? Wherever he pushes, the de- 
fender will push back. Since their forces are equal and con- 
stant, increasing the force along one vector decreases the 
force along another vector. 

Optimization tasks. To follow the fortification strategy, 
D just keeps restoring po to a disk of size 9. To counter 
D's defenses, A needs to find out where they are the weak- 
est. He can observe this wherever D's territory pn is within 
A's range, i.e. contained in the convex hull of pA- So A 
needs to maximize the intersection of his range with D's 
territory. Fig. [5] depicts a position where this is achieved: D 
is under A's siege. It embodies the Fortification Principle, 
that the defender must defend all attack vectors, whereas 
the attacker only needs to select one. For a fast push, A 
randomly selects an attack vector, and waits for D to push 
back. Strengthening D's defense along one vector weakens 
it along another one. Since all of D's territory is within A's 
range, A sees where D's defense is the weakest, and launches 
the next attack there. In contrast, D's range is initially lim- 
ited to his own disk shaped territory. So D only "feels" A's 
pushes when his own borders move. At each step, A pushes 
at D's weakest point, and creates a deeper dent. A does 
enter into D's range, but D's fortification strategy makes 
no use of the information that could be obtained about A. 
The number of steps needed to decrease pn below 9 depends 
on how big are the forces and how small are the contested 
areas. 



3.2.2 Adaptation strategy 

What can D do to avoid the unfavorable outcome of the 
fortification strategy? The idea is that he should learn to 
know his enemy: he should also try to shape his territory 
to maximize the intersection of his range with A's territory. 
D can lure A into his range simply letting A dent his terri- 
tory. This is the familiar honeypot approach, illustrated on 
Fig. 6. Instead of racing around the border to push back 
against every attack, D now gathers information about A's 
next moves within his range. For this, he sacrifices a little 
bit of his territory, as a bait (the "honey") for A. If A sticks 
with his strategic preferences, he will accept D's sacrifice, 
and enter into D's range more and more. Fig. 7 depicts a 



further step in D's strategic development, where he does not 
just passively wait for A to enter the honeypot, but actively 
herds A into it. This is the sampling strategy. Formally, it 
is characterized by D's higher valuation for the information 
about A, than for the territory alone. This is refiected in 
the fact that D's territory gradually evolves into a shape op- 
timized for information gathering. In the cyber wars of the 
day, the sampling strategy is emerging, e.g., among the re- 
searchers who have gone beyond luring some bots into some 
sandboxed computers, and hijacked parts of botnets from 
their owners, for the sole purpose of research [461 145j . Con- 
tinuing in this direction leads to the long term strategy of 
adaptation, depicted on Fig. 8, where all of D's strategic val- 
uation is assigned to the information about the opponent. 
Here A's attacks are actively observed and prevented; the 
territory is maintained as a side effect of keeping the oppo- 
nent localized. In the long term, D wins. The proviso is 
that D has enough territory to begin with. A simplifying 
assumption of the presented model is that A blindly sticks 
with his valuation of the territory, leading him to accept all 
baits. Reality is, of course, not so simple, and A's strategy 
also allows various refinements. However, to achieve his goal 
of stealing D's assets, A cannot avoid entering D's range al- 
together. D, on the other hand, cannot allow that the size 
of his territory drops below 9. Respecting these asymmetric 
constraints, both players' strategy refinements will evolve 
methods to trade territory for information, making increas- 
ingly efficient use of both. 

A formalism for a mathematical analysis of this game is 
sketched in the Appendix. 



3.3 What does all this mean for security? 

The presented toy model provides a very crude picture of the 
evolution of defense strategies from fortification to adapta- 
tion. Intuitively, Fig. 5 can be viewed as a fortress under 
siege, whereas Fig. 8 can be interpreted as a macrophage 
localizing an invader. The intermediate pictures show the 
adaptive immune system luring the invader and sampling 
his chemical markers. 

But there is nothing exclusively biological about the adap- 
tation strategy. Figures 5-8 could also be viewed entirely 
in the context of Figures 3-4, and interpreted as the transi- 
tion from the medieval defense strategies to modern political 
ideas. Fig. 8 could be viewed as a depiction of the idea of 
"preemptive siege": while the medieval rulers tried to keep 



their enemies out of their fortresses, some of the modern ones 
try to keep them in their jails. The evolution of strategic 
thinking illustrated on Figures 5-8 is pervasive in all realms 
of security, i.e. wherever the adversarial behaviors are a 
problem, including cyber-security. 

And although the paradigm of keeping an eye on your ene- 
mies is familiar, the fact that it reverts the odds of security 
and turns them in favor of the defenders does not seem to 
have received enough attention. It opens up a new game 
theoretic perspective on security, and suggests a new tool 
for it. 



4. A BRAND NEW SECURITY PARADIGM: 

APPLIED SECURITY BY OBSCURITY 
4.1 Gaming security basics 

Games of information. In games of luck, each player 
has a type, and some secrets. The type determines player's 
preferences and behaviors. The secrets determine player's 
state. E.g., in poker, the secrets are the cards in player's 
hand, whereas her type consists of her risk aversion, her 
gaming habits etc. The imperfect information means that 
all players' types are a public information, whereas their 
states are unknown, because their secrets are private. In 
games of incomplete information, both players' types and 
their secrets are unknown. The basic ideas and definitions 
of the complete and incomplete informations in games go all 
the way back to von Neumann and Morgenstern 48 . The 
ideas and techniques for modeling incomplete information 
are due to Harsanyi [20], and constitute an important part 
of game theory [5171 [T51 H]. 

Security by secrecy. If cryptanalysis is viewed as a game, 
then the algorithms used in a crypto system can be viewed 
as the type of the corresponding player. The keys are, of 
course, its secrets. In this framewrok, Claude Shannon's 
slogan that "the enemy knows the system" asserts that crypt- 
analysis should be viewed as a game of imperfect informa- 
tion. Since the type of the crypto system is known to the 
enemy, it is not a game of incomplete information. Another 
statement of the same imperative is the Kerckhoffs' slogan 
that "there is no security by obscurity". Here the obscurity 
refers to the type of the system, so the slogan thus suggests 
that the security of a crypto system should only depend on 
the secrecy of its keys, and remain secure if its type is known. 
In terms of physical security, both slogans thus say that the 
thief should not be able to get into the house without the 
right key, even if he knows the mechanics of the lock. The 
key is the secret, the lock is the type. 

Security by obscurity. And while all seems clear, and we 
all pledge allegiance to the Kerckhoffs' Principle, the prac- 
tices of security by obscurity abound. E.g., besides the locks 
that keep the thieves out, many of us use some child-proof 
locks, to protect toddlers from dangers. A child-proof lock 
usually does not have a key, and only provides protection 
through the obscurity of its mechanism. 

On the cryptographic side, security by obscurity remains 
one of the main tools, e.g., in Digital Rights Management 



(DRM), where the task is to protect the digital content from 
its intended users. So our DVDs are encrypted to prevent 
copying; but the key must be on each DVD, or else the DVD 
could not be played. In order to break the copy protection, 
the attacker just needs to find out where to look for the key; 
i.e. he needs to know the system used to hide the key. For a 
sophisticated attacker, this is no problem; but the majority 
is not sophisticated. The DRM is thus based on the second- 
hand but almost-new paradigm from the preceding section: 
the DVD designers study the DVD users and hide the keys 
in obscure places. From time to time, the obscurity wears 
out, by an advance in reverse engineering, or by a lapse of 
defenders attentiorj^. Security is then restored by analyzing 
the enemy, and either introducing new features to stall the 
ripping software, or by dragging the software distributors to 
court. Security by obscurity is an ongoing process, just like 
all of security. 

4.2 Logical complexity 

What is the difference between keys and locks? The 

conceptual problem with the Kerckhoffs Principle, as the re- 
quirement that security should be based on secret keys, and 
not on obscure algorithms, is that it seems inconsistent, at 
least at the first sight, with the Von Neumann architecture of 
our computers, where programs are represented as data. In 
a computer, both a key and an algorithm is a string of bits. 
Why can I hide a key and cannot hide an algorithm? More 
generally, why can I hide data, and cannot hide programs? 

Technically, the answer boils down to the difference between 
data encryption and program obfuscation. The task of en- 
cryption is to transform a data representation in such a way 
that it can be recovered if and only if you have a key. The 
task of obfuscation is to transform a program representation 
so that the obfuscated program runs roughly the same as the 
original one, but that the original code (or some secrets built 
into it) cannot be recovered. Of course, the latter is harder, 
because encrypted data just need to be secret, whereas an 
obfuscated program needs to be secret and and to run like 
the original program. In [5], it was shown that some pro- 
grams must disclose the original code in order to perform 
the same function {and they disclose it in a nontrivial way, 
i.e. not by simply printing out their own code). The theory 
here confirms the empiric evidence that reverse engineering 
is, on the averag43, effective enough that you don't want 
to rely upon its hardness. So it is much easier to find out 
the lock mechanism, than to find the right key, even in the 
digital domain. When they say that there is no security by 
obscurity, security practitioners thus usually mean that re- 

^The DVD Copy Scramble System (CSS) was originally re- 
verse engineered to allow playing DVDs on Linux computers. 
This was possibly facilitated by an inadvertent disclosure 
from the DVD Copy Control Association (CAA). DVD CAA 
pursued the authors and distributors of the Linux DeCSS 
module through a series of court cases, until the case was 
dismissed in 2004 16 . Ironically, the cryptography used in 
DVD CSS has been so weak, in part due to the US export 
controls at the time of design, that any computer fast enough 
to play DVD s could find the key by brute force within 18 
seconds [43]. This easy cryptanalytic attack was published 
before DeCSS, but seemed too obscure for everyday use. 
^However, the International Obfuscated C Code Contest [23] 
has generated some interesting and extremely amusing work. 



verse engineering is easy, whereas cryptanalysis is hard, and 
provides more durable security guarantees^ 

One-way programming? Modern cryptography is based 
on one way functions, which are easy to compute, but hard 
to invert. Secure systems are designed to use the easy direc- 
tion of one-way functions, whereas the attacks must invert 
them, i.e. compute the hard direction. A high level view 
thus displays security as a process where the attackers pro- 
gram attack algorithms in response to the algorithms of the 
system that they attack, whereas the defenders program sys- 
tem algorithms in response to some attacks. The question is 
now whether systems can be designed in such a way to make 
the defenders' programming tasks easy, and the attackers' 
programming tasks hard. Can we lift the idea of one-way 
functions to one-way programming? 

Let us take a closer look. How can we make attacker's task 
harder? Since obfuscation is hard and reverse engineering 
is easy, we assume that the system code is accessible to the 
attacker, and that the attack code is accessible to the de- 
fender. This assumption is supported by the current security 
practices, where the attacker communities reverse engineer 
their target systems, and the security researchers decompile 
malware. But even when the code is completely transpar- 
ent, the attacker's task of uncovering vulnerabilities of the 
system remains nontrivial. And even with a trove of de- 
tected vulnerabilities, the attacker still needs to design an 
effective attack algorithm to exploit them. The hard part 
of an attacker's job is thus the logical task to analyze the 
system, and to design and implement an attack algorithm. 
The logical complexity of such tasks is different from the 
computational complexity of the system and the attack al- 
gorithms. Indeed, a computationally easy algorithm may be 
logically hard to construct, even when it can be expressed 
by a relatively succinct program, like e.g. [2]; whereas an 
algorithm that requires a minor logical effort to construct 
may, of course, require a great computational effort to run. 

The different roles of the computational and the logical com- 
plexities in security can perhaps be pondered on the follow- 
ing example. In modern cryptography, a system C would 
be considered very secure if an attack algorithm Ac on it 
would yield a proof that P = NP. But how would you feel 
about a crypto system L such that an attack algorithm Al 
would yield a proof that P 7^ A'^P? What is the difference 
between the reductions 

Ac =^ P = NP and =^ P / AP ? 

The security of the system C is based on the computational 
complexity of the AP problems. The security of the system 
L is based on the logical complexity of proving P 7^ AP. 
Most computer scientists believe that P 7^ AP is true. If 
P = AP is thus false, then no attack on the system C can 
exist, whereas an attack on the system L may very well be 
possible. So the security of the system L may very well based 
on the obscurity of the proof of P 7^ AP, which most likely 
exists, but is very hard to find. The best minds of mankind 
have spent many years looking for this proof, but did not 

^It is interesting to note that one of the initial ideas for 
public key crypto system, suggested in [TH] , was to partially 
evaluated a symmetric encryption module over its key, and 
to publish its obfuscation as a public encryption module. 



manage to find it. Yet an attack Al, together with the se- 
curity reduction Al => P 7^ AP, would provide such a 
proof. This attack would probably be welcomed with admi- 
ration and gratitude. In fact, the system L is secure enough 
to protect a bank account with a $1,000,000, since proving 
(or disproving) P 7^ AP is worth a Clay Institute Mille- 
nium Prize of $ 1,000,000. If an attacker takes your money 
from the bank account, he will leave you with a proof worth 
much more. So the logical complexity of the system L pro- 
vides enough obscurity for a significant amount of security! 

But what is logical complexity? Computational com- 
plexity of a program tells how many computational steps 
(counting them in time, memory, state changes, etc.) does 
the program take to transform its input into its output. Log- 
ical complexity is not concerned with the execution of the 
program, but with its logical construction. Intuitively, if 
computational complexity of a program counts the number 
of computational steps needed to execute it on an input of a 
given length, its logical complexity should count the number 
of computational steps needed to derive that program from 
some given programming knowledge. For instance, while 
the computational complexity of an attack on a crypto sys- 
tem is the the number of computational steps that it re- 
quires to extract some information about the plaintext from 
the cyphertext, the logical complexity of that attack is the 
number of logical steps needed to find that attack algorithm 
from a given description of the system algorithm. In other 
words, the logical complexity of an attack on a crypto sys- 
tem is the computational complexity of the task of finding 
a counterexample for the security claim of that systemQ 

The idea is thus to define logical complexity of an algorithm 
A as the computational complexity of the fastest construc- 
tion algorithm Pa that outputs A, given some algorithmic 
knowledge as the input. The problem in formalizing this 
is that the fastest program that outputs A is the program 
print A, which does not need any input, since A is hard- 
wired in it. To assure that A is constructed in a nontrivial 
way, we need to look for a constructor Pa that is the fastest 
among the algorithms that can be implemented by programs 
significantly shorter than A itself. 

This brings us into the realm of algorithmic information 
theory [1 11 1281 [29] . where similar concepts, with slight varia- 
tions, have been proposed under a variety of different names. 
Bennett's logical depth [7] seems to be the the closest. In 
Bennett's original formulation, logical depth is defined as 
a complexity measure assigned to data, or to observations, 
although Bennett's fascinating analyses also assign logical 
depth to genes and to organisms, as the time that it took 
them to evolve [6]. The step from logical depth of data 
to logical complexity of algorithms boils down to the view 
of algorithms-as-programs-as-data, originating from Godel's 
enumeration of recursive functions as numbers, and from 
Kleene's recursive indices. Towards a brief, somewhat over- 
simplified, but hopefully not misleading account of these 
formalisms, consider the partial algebraic theory with two 

^Such claims are usually stated in the form: "For all prob- 
abilistic polynomial-time Turing machines that the attacker 
may use, his advantage is not greater than [some formula]". 
The logical complexity of the attack is the computational 
complexity of attacker's task of finding that attack. 



sorts, 

• N, representing the data, say as numbers, and 

• M, representing a family of algorithms, viewed as par- 
tial functions N* ^ N, say those that can be realized 
by Turing machine^, 

given together with the mappings 



M^:;;;;2_II^N (i) 
{-} 

which make them isomorphic, i.e. 

{^Afi} = and ''{n}^ = n 

This means not just that every machine M € M can be 
encoded as a number '~M~' G N, but also that any number 
n G N can be viewed as a program corresponding to the 
machine {n} G M, and executed on data. In addition, there 
are also 

• a universal composition machine U G M, such that for 
all P,Q eM and all a; G N 

U(rP^,rQn,n) = P(Q(n)) (2) 

holds whenever either side is defined, 

• a time counter T G M, where T{'~M~' , n) denotes the 
number of steps that the machine M takes before it 
halts on the input n, and 

• a length function ^ : N — )■ N, assigning a to each piece 
of data its length. 

Towards formalizing the above idea of logical complexity, we 
now define the algorithmic distance C{A, B) between the al- 
gorithms y4, _B G M to be the length of the shortest program 
p that inputs the code ^A~^ and outputs the code ^B^: 

C{A,B) = /\ £{p) (3) 

{p}{rAn)=rB-i 

The logical distance T>{A, B) is now the shortest time that 
it takes to compute ^B^ from '"A"' by one of the shortest 
programs: 

V{A,B) = f\ T{p,rA^) (4) 

{p}(rAn)=rBn 
e(p)=C{A,B) 

Remarks. Algorithmic distance is based on the relativized 
version of Solomonoff's and Kolmogorov's definitions of com- 
plexity [411 126] . Logical distance is based on a relativized 
and simplified version of Bennett's logical depth [7]. It is 
simplified in the sense that it does not take into account 
the possibility that slightly longer programs may run signif- 
icantly faster. To capture this, the developed definitions of 

^More precisely, they are realized as self-delimiting Turing 
machines, which a llow s them to receive multiple arguments 
on the same tape [49| . 



logical depth are parametrized over the difference in length 
between the fastest and the shortest programs. In some ap- 
plications, this is an important technical detail, assuring the 
stability of the definition. In the current presentation, which 
is mainly conceptual, it would just complicate the definition. 
A conceptual detail which is also omitted is that the univer- 
sal composition machine, and the programs in ^ and Q 
need to be homomorphisms with respect to certain logical 
operations on programs. This additional requirement seems 
essential for the envisioned applications of logical distance 
as a tool of security by obscurity. Nevertheless, such matters 
must be left for future work. The path towards program and 
specification frameworks that would take into account the 
logical distances of algorithms, and advance the idea of one- 
programming, requires examining the diverse refinements of 
the basic idea of the Godel-Kleene program encodings that 
in the meantime emerged from the theory and the extensive 
experience of program development [211 1101 1151 1391 1321 138] . 

Logical security. The idea of logical security is to make the 
derivation of an attack algorithm from a system algorithm 
logically complex. Formally, a system S would thus be log- 
ically security if the distance 'D{S, A) is large for all attacks 
A on S. On the other hand, since the distance is obviously 
not symmetric, V{A, S) may be small, meaning that it may 
be easy to derive an improved system algorithm S from an 
undesired algorithm A. This connects logical security with 
the idea of one-way programming. 

At this point, the reader may object that it does not make 
much sense to call T> a distance when ©(A, S) is generally 
different from ©(S', A). A possible answer to appeal to the 
spatial intuition that S lays lower than A, so that getting 
from S to A requires stranuous climbing, and from A to S 
only breaking. But it gets worse. While algorithmic distance 
satisfies the triangle law 

CiP,Q)+CiQ,R) > C(P,R) (5) 

realized by the universal composition machine, logical dis- 
tance generally does not satisfy this law, since there may 
be a short but slow algorithm to construct R from P, but 
it does not have to go through Q, and all algorithms to 
construct Q from P and R from Q may be long but fast. 
Nevertheless, logical distance can be easily shown to satisfy 
the weaker law 

V{P,Q) + V{P AQ,R) > V{P,R) (6) 

where P A Q denotes the parallel composition of P and Q, 
i.e. an algorithm that satisfies PA Q{x,y) — {u,v) if and 
only if P{x) = u and Q{x) — v, for all x,y,u,v. One is 
tempted to call this law "subbayesian", since it echoes the 
Bayes' law for conditional probabilities in the form 

Pr(r I p) = Pr(r | q A p) ■ Pr{q \ p) 

where p, q, r denote events. At any rate, we can now to use 
([6]) to bound the logical complexity A) of constructing 
an attack yl on a system S. For the particular case of the 
system L, given with a security reduction Al P 7^ A^P, 
we first of all assume that we are given an effective algo- 
rithnfl to transform any attack Al to a, proof of P / A^P. 

^The constructivist imperative that an implication like 
Al P 7^ A''P should be supported by an effective algo- 



Denote the time complexity of the shortest such algorithm 
by 

d = V{Al, P^NP) 

where we abuse notation and write P 7^ NP for the algo- 
rithm that outputs a proof of the statement P 7^ NP. Now 
if the system L itself does not allow shortening of the proof 
Al=^ P ^ NP, i.e. if d > V{L aAl, P + NP, then we 
can get 

V{L, Al) + d > V{L, P^NP) 

as a substitution instance of ([6|. Going back to security by 
obscurity, this means that, although L may be vulnerable to 
a computationally easy attack Al, constructing this attack 
may be logically hard, nearly as hard as deriving a proof of 
P / NP from it. 

Logical complexity of randomized algorithms. While 
logical complexity as a tool and resource for security will 
clearly be built upon the deep and interesting results of 
algorithmic information theory, it should be noted that a 
realistic model of attacker's logical practices will require al- 
gorithmic information theory of randomized computation, 
since the algorithms used in security tend to be random- 
ized. The encoding in will thus not be strict but only 
approximate, with replaced with 

Pr((7(rpn^,rQn^,x) = P(Q(a;))) > e (7) 



4.3 Logical complexity of gaming security 

Randomized logical complexity brings us back to security as 
a game of incomplete information. In order to construct a 
strategy each player in such a game must supplement the 
available informations about the opponent by some beliefs. 
Mathematically, these beliefs have been modeled, ever since 
[20| . as probability distributions over opponent's payoff func- 
tions. More generally, in games not driven by payoffs, beliefs 
can be modeled as probability distributions over the possible 
opponent's behaviors, or algorithms. In terms of random- 
ized logical complexity, such a belief can thus be viewed as an 
approximate logical specification of opponent's algorithms. 

Since both players in a game of incomplete information are 
building beliefs about each other, they must also build be- 
liefs about each other beliefs: A formulates a belief about 
B's belief about A, and B formulates a belief about A^s be- 
lief about B. And so on to the infinity. This is described 
in more detail in the Appendix, and in still more detail in 
dj . These hierarchies of beliefs are formalized as probability 
distributions over probability distributions. The framework 
of approximate logical complexity now captures the way in 
which the players encode their beliefs about each other's 
beliefs. This leads into an algorithmic theory of incomplete 
information, where players' belief hierarchies consist of sam- 
plable probability distributions. 



rithm transforming any proof of the antecedent into a proof 
of the consequence was goes back to Brouwer 47 . Although 
constructivism was deemed impractical by most mathemati- 
cians, functional programming can be viewed as its practical 
realization jl9j i34j i35j . 



5. FINAL COMMENTS 

On games of security and obscurity. The first idea of 
this paper is that security is a game of incomplete informa- 
tion: by analyzing your enemy's behaviors and algorithms 
(subsumed under what game theorists call his type), and by 
obscuring your own, you can improve the odds of winning 
this game. 

This claim contradicts Kerckhoffs' Principle that there is no 
security by obscurity, which implies that security should be 
viewed as a game of imperfect information, by asserting that 
security is based on players' secret data (e.g. cards), and not 
on their obscure behaviors and algorithms. 

I described a toy model of a security game which illustrates 
that security is fundamentally based on gathering and an- 
alyzing information about the type of the opponent. This 
model thus suggests that security is not a game of imper- 
fect information, but a game of incompete information. If 
confirmed, this claim implies that security can be increased 
not only by analyzing attacker's type, but also by obscuring 
defender's type. 

On logical complexity. The second idea of this paper is 
the idea of one-way programming, based on the concept of 
logical complexity of programs. The suggestion is that al- 
gorithmic information theory may be a useful new area for 
security research. It provides a natural conceptual frame- 
work for studying algorithm evolution driven by the battles 
between the attackers and defenders. It also provides some 
concrete technical tools, which raise a possibility of systems 
vulnerable to computationally feasible, but logically unfeasi- 
ble attacks; in other words, the attack algorithms that may 
be easy to run when you know them, but hard to construct 
if you don't know them. Security experts dismiss such ideas, 
mainly because the experience and theory show that algo- 
rithms are never too hard to reconstruct from their obfus- 
cations. But constructing them from other algorithms can 
be genuinely hard. 

On security of profiling. Typing and profiling are frowned 
upon in security. Leaving aside the question whether gath- 
ering information about the attacker, and obscuring the sys- 
tem, might be useful for security or not, these practices re- 
main questionable socially. The false positives arising from 
such methods cause a lot of trouble, and tend to just drive 
the attackers deeper into hiding. 

On the other hand, typing and profiling are technically and 
conceptually unavoidable in gaming, and remain respectable 
research topics of game theory. Some games cannot be 
played without typing and profiling the opponents. Poker 
and the bidding phase of bridge are all about trying to guess 
your opponents' secrets by analyzing their behaviors. Play- 
ers do all they can to avoid being analyzed, and many prod 
their opponents to sample their behaviors. Some games can- 
not be won by mere uniform distributions, without analyzing 
opponents' biases. 

Both game theory and immune system teach us that we 
cannot avoid profiling the enemy. But both the social ex- 
perience and immune system teach us that we must set the 



thresholds high to avoid the false positives that the profiling 
methods are so prone to. Misidentifying the enemy leads 
to auto-immune disorders, which can be equally pernicious 
socially, as they are to our health. 
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APPENDIX 

A. GAMING SECURITY FORMALISM 

Can the idea of applied security by obscurity be realized? To 
test it, let us first make it more precise in a mathematical 
model. I first present a very abstract model of strategic be- 
havior, capturing and distinguishing the various families of 
games studied in game theory, and some families not stud- 
ied. The model is based on coalgebraic methods, along the 
lines of [36]. I will try to keep the technicalities at a min- 
imum, and the reader is not expected to know what is a 
coalgebra. 

A.l Arenas 

Definition 1. A player is a pair of sets A = {Ma,Sa), 
where the elements of Ma represent or moves available to 
A, and the elements of Sa are the states that A may observe. 

A simple response : A B for a player B to a player A 
is a binary relation 

E : Max Sl X Mb ^ {0, 1} 

When E(o,^,/3',b) = 1, we write (a,/3) ^ (/3',6), and say 
that the strategy E at B's state /3 prescribes that B should 
respond to A's move a by the move b and update his state to 
j3' . The set of _B's simple responses to A is written SR(A, B). 

A mixed response $ : A — )■ _B for the player B to the player 
A is a. matrix 

$ : Ma X Sl X Mb ^ [0, 1] 

required to be finitely supported and stochastic in Ma, i.e. 
for every a £ Ma holds 

• ^api3'b ~ holds for all but finitely many /3, /3' and b, 

• E/3/3'6 *a/3/3'b = 1- 

When $a/3/3'b = p we write (a, /3) {13', b), and say that 

p 

the strategy $ at B's state /3 responds to A's move a with 
a probability p by B's move b leading him into the state j3' . 
The set of B's mixed responses to A is written MR(yl, B). 

An arena is a specification of a set of players and a set of 
responses between them. 



Responses compose. Given simple responses E : A — >■ B 
and r : B — > C, we can derive a response (E; F) : A — > C for 
the player C against A by taking the player B as a "man in 
the middle". The derived response is constructed as follows: 

{a,P)^{P',b) (b,'y)^{j',c) 

(a, 7) (7',c) 

Following the same idea, for the mixed responses $ : A — >■ B 
and \l/ : B — >• C we have the composite ("I?; *) : ^ — >■ C with 
the entries 



It is easy to see that these composition operations are asso- 
ciative and unitary, both for the simple and for the mixed 
responses. 

A.2 Games 

Arenas turn out to provide a convenient framework for a 
unified presentation of games studied in game theory [481 
133) . mathematical games l8l, game semantics [TJ [22], and 
some constructions in-between these areas [13]. Here we 
shall use them to succinctly distinguish between the various 
kinds of game with respect to the information available to 
the players. As mentioned before, game theorists usually 
distinguish two kinds of players' information: 

• data, or positions: e.g., a hand of cards, or a secret 
number; and 

• types, or preferences: e.g., player's payoff matrix, or a 
system that he uses are components of his type. 

The games in which the players have private data or posi- 
tions are the games of imperfect information. The games 
where the players have private types or preferences, e.g. be- 
cause they don't know each other's payoff matrices, are the 
games of incomplete information. See [3] for more about 
these ideas, [33] for the technical details of the perfect- 
imperfect distinction, and [18] for the technical details of 
the complete-incomplete distinction. 

But let us see how arenas capture these distinction, and 
what does all that have to do with security. 

A. 2. 1 Games of perfect and complete information 

In games of perfect and complete information, each player 
has all information about the other player's data and pref- 
erences, i.e. payoffs. To present the usual stateless games in 
normal form, we consider the players A and B whose state 
spaces are the sets payoff bimatrices, i.e. 

Sa ^ Sb = (R X R) 

In other words, a state a £ Sa = Sb is a pair of maps 
a = {a^,a^) where is the Ma x Ms-matrix of ^'s pay- 
offs: the entry a^f, is A^s payoff if A plays a and B plays 
b. Ditto for gb- Each game in the standard bimatrix form 
corresponds to one element of both state spaces Sa = Sb. 
It is nominally represented as a state, but this state does not 
change. The main point here is that both A and B know 
this element. This allows both of them to determine the 
best response strategies Ea : B ^ A for A and Es : A ^ B 
for B, in the form 

{b, cr''*) {a^, a) Vx G Ma- cr^f, < crfb 

{a,a^) {a^ ,b) <;=> & Mb ■ (^ay < (^ab 

and to compute the Nash equilibria as the fixed points of 
the composites (E-^;E^) : A ^ A and (E'*;E-^) : B 

B. This is further discussed in [3^. Although the payoff 
matrices in games studied in game theory usually do not 
change, so the corresponding responses fix all states, and 
each response actually presents a method to respond in a 
whole family of games, represented by the whole space of 



payoff matrices, it is interesting to consider, e.g. discounted 
payoffs in some iterated games, where the full force of the 
response formalism over the above state spaces is used. 

A. 2. 2 Games of imperfect information 

Games of imperfect information are usually viewed in ex- 
tended form, i.e. with nontrivial state changes, because 
players' private data can then be presented as their private 
states. Each player now has a set if private positions, Pa 
and Pb, which is not visible to the opponent. On the other 
hand, both player's types, presented as their payoff matrices, 
are still visible to both. So we have 

Sa = Pa X (R X R)*^^''"« 
Sb = Ps X (R X R)*^^''"« 

E.g., in a game of cards, A^s hand will be an element of Pa, 
B's hand will be an element of Pb. With each response, each 
player updates his position, whereas their payoff matrices 
usually do not change. 

A.2. 3 Games of incomplete information 

Games of incomplete information are studied in epistemic 
game theory [201 1301 [3] , which is formalized through knowl- 
edge and belief logics. The reason is that each player here 
only knows with certainty his own preferences, as expressed 
by his payoff matrix. The opponent's preferences and payoffs 
are kept in obscurity. In order to anticipate opponent's be- 
haviors, each player must build some beliefs about the other 
player's preferences. In the first instance, this is expressed 
as a probability distribution over the other player's possi- 
ble payoff matrices. However, the other player also builds 
beliefs about his opponent's preferences, and his behavior 
is therefore not entirely determined by his own preferences, 
but also by his beliefs about his opponent's preferences. So 
each player also builds some beliefs about the other player's 
beliefs, which is expressed as a probability distribution over 
the probability distributions over the payoff matrices. And 
so to the infinity. Harsanyi formalized the notion of players 
type as an element of such information space, which includes 
each player's payoffs, his beliefs about the other player's pay- 
offs, his beliefs about the other player's beliefs, and so on 
[20| . Harsanyi's form of games of incomplete information 
can be presented in the arena framework by taking 

Sa = R^^^'^^^+ASfl 
5s = R^'^"^'''' + ASa 

where -I- denotes the disjoint union of sets, and AX es the 
space of finitely supported probability distributions over X, 
which consists of the maps p : X — >■ [0, 1] such that 

\{x G X\p{x) > 0}\ < oo and ^ p{x) = 1 

xex 

Resolving the above inductive definitions of Sa and Sb, we 
get 

OO 

Sa ^ Sb = («^"^ 
Here the state G Sa is thus a sequence 

A I A A A \ 
O" = (CTq .0"! .'^2 I ■ ■ ■/ 



where af € A'R**^-*^**^ . The even components cr^ repre- 
sent A's payoff matrix, A's behef about B's behef about A's 
payoff matrix, A's belief about B's belief about yl's belief 
about _B's belief about A's payoff matrix, and so on. The 
odd components represent A's belief about B's payoff 

matrix, A's belief about B's belief about A's belief about _B's 
payoff matrix, and so on. The meanings of the components 
of the state £ Sb are analogous. 

A.3 Security games 

We model security processes as a special family of games. 
It will be a game of imperfect information, since the players 
of security games usually have some secret keys, which are 
presented as the elements of their private state sets Pa and 
Pd- The player A is now the attacker, and the player D is 
the defender. 

The goal of a security game is not expressed through payoffs, 
but through "security requirements" Q C P^,. The intuition 
is that the defender D is given a family of assets to protect, 
and the are the desired states, where these assets are 
protected. The defender's goal is to keep the state of the 
game in 0, whereas the attacker's goal is to drive the game 
outside Q. The attacker may have additional preferences, 
expressed by a probability distribution over his own private 
states Pa- We shall ignore this aspect, since it plays no role 
in the argument here; but it can easily be captured in the 
arena formalism. 

Since players' goals are not to maximize their revenues, their 
behaviors are not determined by payoff matrices, but by 
their response strategies, which we collect in the sets RA(j4, D) 
and RA(_D,A). In the simplest case, response strategies 
boil down to the response maps, and we take RA(j4, D) — 
SR{A,D), or RA{A,D) = MR{A,D). In general, though, 
A's and -D's behavior may not be purely extensional, and 
the elements of RA{A, D) may be actual algorithms. 

While both players surely keep their keys secret, and some 
part of the spaces Pa and Pd are private, they may not know 
each other's preferences, and may not be given each other's 
"response algorithms". If they do know them, then both 
defender's defenses and attaker's attacks are achieved with- 
out obscurity. However, modern security definitions usually 
require that the defender defends the system against a fam- 
ily of attacks without querying the attacker about his algo- 
rithms. So at least the theoretical attacks are in principle 
afforded the cloak of obscurity. Since the defender D thus 
does not know the attacker A's algorithms, we model se- 
curity games as games of incomplete information, replacing 
the player's spaces of payoff matrices by the spaces RA(A, D) 
and RA(D, A) of their response strategies to one another. 

Like above, A thus only knows Pa and RA(D, A) with cer- 
tainty, and D only knows Pd and RA(A, D) with certainty. 
Moreover, A builds his beliefs about D's data and type, as a 
probability distribution over Pd x RA(A,Z)), and D builds 
similar beliefs about A. Since they then also have to build 
beliefs about each other's beliefs, we have a mutually recur- 
rent definition of the state spaces again: 

Sa = (Pa X RA(D,A)) + A^D 

Sd = [Pd X Rh{A,D)) + AS a 



Resolving the induction again, we now get 

oo 
oo 

Sd = Yl X RA(A, D)) x A^'+^ {Pa x RA{D, A)) 

Defender's state P G Sd is thus a sequence /3 = (/3o , Pi, 132, . . .), 
where 

• l3o = {Po,Po'^) €Pd X RA{A,D) consists of 

— D's secrets £ Pd, and 

— D's current response strategy 0^'^ G RA(A, D) 

• /3i e A(Pa X RA(D, A)) is D's belief about A's secrets 
and her response strategy; 

• A^{Pd X RA(A,D)) is D's belief about A's belief 
about D's secrets and his response strategy; 

• /33 e A^{Pa X RA(D, A)) is D's belief about A's belief 
about D's beliefs, etc. 

Each response strategy E : A ^ D prescribes the way in 
which D should update his state in response to A's observed 
moves. E.g., if RA(D, A) is taken to consist of relations in 
the form A : M+ x Ma -> {0, 1}, where M+ is the set of 
nonempty strings in Md, then D can record the longer and 
longer histories of A's responses to his moves. 

Remark. The fact that, in a security game, A's state space 
Sa contains RA(D, A) and RA(A, D) means that each player 
A is prepared for playing against a particular player D; while 
D is prepared for playing against A. This reflects the sit- 
uation in which all security measures are introduced with 
particular attackers in mind, whereas the attacks are built 
to attack particular security measures. A technical conse- 
quence is that players' state spaces are defined by the in- 
ductive clauses, which often lead to complex impredicative 
structures. This should not be surprising, since even infor- 
mal security considerations often give rise to complex belief 
hierarchies, and the formal constructions of epistemic game 
theory |20l 1301 117) seem like a natural tool to apply. 

A.4 The game of attack vectors 

We specify a formal model of the game of attack vectors as 
a game of perfect but incomplete information. This means 
that the players know each other's positions, but need to 
learn about each other's type, i.e. plans and methods. The 
assumption that the players know each other's position could 
be removed, without changing the outcomes and strategies, 
by refining the way the model of players' observations. But 
this seems inessential, and we omit it for simplicity. 

Let O denote the unit disk in the real plane. If we assume 
that it is parametrized in the polar coordinates, then O = 
{0} U (0, 1] X R/27rZ, where R/27rZ denotes the circle. Let 
7^ C r2 be an open domain in the real plane. Players' 
positions can then be defined as continuous mappings of O 
into 72,, i.e. 

Pa = Pd = 11° 



The rules of the game will be such that the attacker's and 
the defender's positions pa,Pd £ 'R-'^ always satisfy the con- 
straint that 

p°Arip°D = 

where p° denotes the interior of the image oi p : O ^ TZ. 
The assumption that both players know both their own and 
the opponent's positions means that both state spaces Sa 
and Sd will contain Pa x Pd as a component. The state 
spaces are thus 

Sa = {Pa x Pd x RA{D, A)) + ASd 
Sd = {Pa x Pd x RA{A, D)) + ASd 

To start off the game, we assume that the defender D is 
given some assets to secure, presented as an area Q C TZ. 
The defender wins as long as his position po £ Pd is such 
that Q C p^. Otherwise the defender loses. The attacker's 
goal is to acquire the assets, i.e. to maximize the area QDPa- 

The players are given equal forces to distribute along the 
borders of their respective territories. Their moves are the 
choices of these distributions, i.e. 

Ma = Md = A (90) 

where dO is the unit circle, viewed as the boundary of O, 
and A{dO) denotes the distributions along dO, i.e. the 
measurable functions m : dO — !■ [0, 1] such that J^^ m — 1. 

How will D update his space after a move? This can be 
specified as a simple response T. : A ^ D. Since the point 
of this game is to illustrate the need for learning about the 
opponent, let us leave out players' type information for the 
moment, and assume that the players only look at their 
positions, i.e. Sa ~ Sd ~ Pa x Pd- To specify T, : A B, 
we must thus determine the relation 

{mA,PA,PD) A {PAyPDy-rnD) 

for any given niA G Ma, Pa £ Pa, and pd £ Pd- We 
describe the updates p'a and p'd for an arbitrary ruD, and 
leave it to D to determine which m_DS are the best responses 
for him. So given the previous positions and both player's 
moves, the new position p'a will map a point on the boundary 
of the circle, viewed as a unit vector x £ O into the vector 
p'a{^) in 7^ C as follows. 



• If for all y £ O and for all s £ [0,mA(2f)] and all t £ 
[0,mD{y)] holds {1 + s)pa{x) / {l + t)pD{y) then set 

^/ 1 + mA{x) ^ 
Pa{x) = 2 P-^*.^) 

• Otherwise, let jT £ O, s £ [0, mA{x)] and t £ [0, mD{y)] 
be the smallest numbers such that (1 + s)pa(x) = (1 + 
t)pD{y)- Writing m^(£) — mA{x) — s and m'D{if) = 
mD{y) — t, we set 

pa(^) = ^^■pa{^) + f^-PD{y) 

This means that the player A will push her boundary by 
mA{x) in the direction pa{x) if she does not encounter D 
at any point during that push. If somewhere during that 
push she does encounter D's territory, then they will push 
against each other, i.e. their push vectors will compose. 
More precisely, A will push in the direction pa {x) with the 
force m'A{x) — mA{x) — s, that remains to her after the 
initial free push by s; but moreover, her boundary will also 
be pushed in the direction PD{y) by the boundary of D's 
territory, with the force m'D{y) = mD{y) — t, that remains 
to D after his initial free push by t. Since D's update is 
defined analogously, a common boundary point will arise, 
i.e. players' borders will remain adjacent. When the move 
next, there will be no free initial pushes, i.e. s and t will be 
0, and the update vectors will compose in full force. 

How do the players compute the best moves? Attacker's 
goal is, of course, to form a common boundary and to push 
towards O, preferably from the direction where the defender 
does not defend. The defender's goal is to push back. As 
explained explained in the text, the game is thus resolved on 
defender's capability to predict attacker's moves. Since the 
territories do not intersect, but A's moves become observable 
for D along the part of the boundary of A's territory that 
lies within the convex hull of D's territory, D's moves must 
be selected to maximize the length of the curve 

dp A n conv(pD) 

This strategic goal leads to the evolution described infor- 
mally in Sec. 13.21 



